1. Technical Field
The invention relates generally to digital image processing, and particularly to a programmable image transform processor for digital image processing.
2. Related Art
In photographic cameras, the image-forming light is sensed and recorded directly on film. Unlike photographic cameras, the electronic still camera uses an electronic image sensor to sense the image-forming light and a separate recording medium to record and store the picture. Because the electronic still camera uses digital technology, the electronic still camera is a type of digital camera.
Typically the electronic image sensor in a digital camera is a solid-state device such as a charge-coupled device (CCD), charge injected device (CID) or a complimentary metal oxide semiconductor (CMOS) device. The image sensor connects to electronic interface circuitry which connects to a storage device and, optionally, to a display. A typical image sensor has many cells or pixels arranged along vertical and horizontal dimensions in a matrix. In response to light, the cells generate a charge or voltage which represents image information. The image sensor senses an image and stores image information, i.e., a charge or voltage, corresponding to the sensed light in the cells. Image sensors are made in many sizes such as, e.g., 400×300, 640×480, 1024×768 and 4096×4096 pixels. The image information stored in the cells is output serially from the image sensor using an arrangement of shift registers. The shift registers are arranged along vertical and horizontal dimensions and are coupled to the cells. The cells and shift registers require timing, or clock signals, having specific timing requirements, to output the image information. Each type of image sensor has its own unique timing requirements. Typically, a single image sensor requires many clock signals to control the flow of image information in both the horizontal and vertical dimensions. The clock signals must be synchronized. For example, to output image information from a 640×480 CCD requires 480 vertical shifts and 640 horizontal shifts for each vertical shift. Within a single dimension, the clock signals to control the flow of image information have different phases that must be synchronized. Furthermore, shifting the information out of the image sensor requires timing signals to synchronize the image sensor's operation with an analog signal processor (ASP) and an analog-to-digital (A/D) converter.
The image information sensed by each cell is also called a pixel. For example, a 640×480 CCD has about 307,200 pixels. After being converted to digital form, the image information (image data) is stored in a memory, typically an image memory. Image sensors having a larger numbers of cells produce higher quality images; however, the more pixel information that is available relates to the amount of processing and memory resources required to process the pixel information.
Typically, a digital signal processor processes the image data to improve the quality of the image. Various algorithms well-known in the art are used to improve the image quality of the image data. Because there is such a large amount of image data, the image data may be compressed before storage in a storage medium or memory.
Color imaging increases the complexity of processing the image data. In one method, the image sensor has a geometric arrangement of cells to respond to three colors, e.g., red, green and blue. Since each cell senses a particular color, various algorithms are used to interpolate the missing color information. Alternatively, two or more image sensors having different color sensitivity may be utilized and the image information combined.
In digital cameras, processing the data takes time. Analog image data from the image sensor is processed via the analog signal processor, converted into image data by the analog-to-digital converter and stored in memory. Furthermore, a digital signal processor processes the raw image data to improve the quality of the image. For color images that utilize a single image sensor, “missing” pixel data values must be interpolated and require even more processing time. Still images are further processed to compensate and correct for other errors introduced by the optical system and the image sensor. The compression of the image data adds even more time. The time required to acquire, process and compress the image data causes an unacceptable delay when acquiring consecutive images. The delay can take several seconds. This delay is a problem for photographers who need a continuous shooting capability to photograph a sequence of images in quick succession. Therefore a process and apparatus are needed to reduce the delay between consecutive pictures.
Typically, a digital camera has hardware that implements a single digital image processing procedure or algorithm. If the procedure is changed, the hardware must be redesigned, which is time consuming and expensive. Therefore, there is a need in the art for a digital image processing procedure or device that is easily and quickly modified and that supports numerous digital signal processing procedures using the same hardware. The digital image processing procedure or device should also minimize the processing time to allow consecutive pictures to be taken in quick succession.
In addition, depending on the environmental factors, such as lighting, the image processing algorithm should be selected or modified to produce the desired image quality. Furthermore, there is a need to dynamically modify the image processing algorithm during the image acquisition process.
As the size of the image sensors increases, the amount of image information to be processed increases. In addition, as image processing algorithms become increasingly sophisticated, complex processing of the image data consumes more time. Therefore, there is a need to reduce the image processing time.